System for observing and analyzing customer opinion

ABSTRACT

A system for observing and analyzing customer opinions is disclosed. The system includes one or more cameras, a focus tracking module, an observation module, an interpretation module, and a rating module. The focus tracking module uses the one or more cameras to determine a product having a focus of a person, the product having a plurality of characteristics. The observation module uses the one or more cameras to monitor one or more physical, physiological, or behavioral characteristics of the person with respect to the product. The interpretation module determines an opinion of the person regarding the product. The rating module adjusts one or more ratings of the product in response to the interpretation module determining the opinion of the person regarding the product.

FIELD

The subject matter disclosed herein relates to observing and analyzingcustomer opinion and more particularly relates to real-time observationand analysis of retail customer opinions to products.

BACKGROUND

Retail stores routinely decide what to sell and stock product inventorybased on a combination of past sales and future projections based oncustomer product surveys. Customer surveys of new products are oftenconducted in a controlled environment that differ from an actual retailshopping experience. Products that are popular with consumers in acontrolled environment may not sell well in an actual store, or viceversa. A variety of factors besides the actual physical characteristicsof a product may influence whether a retail customer will purchase it.Among other things, the price, marketing, and location of a productwithin a store may all influence a retail consumer's purchasingdecision.

SUMMARY

A system for observing and analyzing customer opinions is disclosed. Thesystem includes one or more cameras, a focus tracking module, anobservation module, an interpretation module, and a rating module. Thefocus tracking module uses the one or more cameras to determine aproduct having a focus of a person, the product having a plurality ofcharacteristics. The observation module uses the one or more cameras tomonitor one or more physical, physiological, or behavioralcharacteristics of the person with respect to the product. Theinterpretation module determines an opinion of the person regarding theproduct. The rating module adjusts one or more ratings of the product inresponse to the interpretation module determining the opinion of theperson regarding the product.

A method for observing and analyzing customer opinions is disclosed. Themethod includes using one or more cameras to determine a product havinga focus of a person, using the one or more cameras to monitor one ormore physical, physiological, or behavioral characteristics of theperson with respect to the product, determining an opinion of the personregarding the product, and adjusting one or more ratings of the productin response to determining the opinion of the person regarding theproduct.

A computer program product for observing and analyzing customer opinion,the computer program product comprising a computer readable storagemedium having program instructions embodied therewith, the programinstructions readable/executable by a processor to cause the processorto use one or more cameras to determine a product having a focus of aperson, use the one or more cameras to monitor one or more physical,physiological, or behavioral characteristics of the person with respectto the product, determine an opinion of the person regarding theproduct, and adjust one or more ratings of the product in response todetermining the opinion of the person regarding the product.

BRIEF DESCRIPTION OF THE DRAWINGS

In order that the advantages of the embodiments of the invention will bereadily understood, a more particular description of the embodimentsbriefly described above will be rendered by reference to specificembodiments that are illustrated in the appended drawings. Understandingthat these drawings depict only some embodiments and are not thereforeto be considered to be limiting of scope, the embodiments will bedescribed and explained with additional specificity and detail throughthe use of the accompanying drawings, in which:

FIG. 1 is a schematic block diagram illustrating one embodiment of asystem for observing and analyzing customer opinion in accordance withthe present invention;

FIG. 2 is a schematic block diagram illustrating one embodiment of anapparatus for observing and analyzing customer opinion in accordancewith the present invention;

FIG. 3 is a schematic flow chart diagram illustrating one embodiment ofa method for observing and analyzing customer opinion in accordance withthe present invention; and

FIG. 4 is a schematic flow chart diagram illustrating another embodimentof a method for observing and analyzing customer opinion in accordancewith the present invention.

FIG. 5 is a schematic flow chart diagram illustrating yet anotherembodiment of a method for observing and analyzing customer opinion inaccordance with the present invention.

DETAILED DESCRIPTION

Reference throughout this specification to “one embodiment,” “anembodiment,” or similar language means that a particular feature,structure, or characteristic described in connection with the embodimentis included in at least one embodiment. Thus, appearances of the phrases“in one embodiment,” “in an embodiment,” and similar language throughoutthis specification may, but do not necessarily, all refer to the sameembodiment, but mean “one or more but not all embodiments” unlessexpressly specified otherwise. The terms “including,” “comprising,”“having,” and variations thereof mean “including but not limited to”unless expressly specified otherwise. An enumerated listing of itemsdoes not imply that any or all of the items are mutually exclusiveand/or mutually inclusive, unless expressly specified otherwise. Theterms “a,” “an,” and “the” also refer to “one or more” unless expresslyspecified otherwise.

Furthermore, the described features, advantages, and characteristics ofthe embodiments may be combined in any suitable manner. One skilled inthe relevant art will recognize that the embodiments may be practicedwithout one or more of the specific features or advantages of aparticular embodiment. In other instances, additional features andadvantages may be recognized in certain embodiments that may not bepresent in all embodiments.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (“RAM”), aread-only memory (“ROM”), an erasable programmable read-only memory(“EPROM” or Flash memory), a static random access memory (“SRAM”), aportable compact disc read-only memory (“CD-ROM”), a digital versatiledisk (“DVD”), a memory stick, a floppy disk, a mechanically encodeddevice such as punch-cards or raised structures in a groove havinginstructions recorded thereon, and any suitable combination of theforegoing. A computer readable storage medium, as used herein, is not tobe construed as being transitory signals per se, such as radio waves orother freely propagating electromagnetic waves, electromagnetic wavespropagating through a waveguide or other transmission media (e.g., lightpulses passing through a fiber-optic cable), or electrical signalstransmitted through a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

Many of the functional units described in this specification have beenlabeled as modules, in order to more particularly emphasize theirimplementation independence. For example, a module may be implemented asa hardware circuit comprising custom VLSI circuits or gate arrays,off-the-shelf semiconductors such as logic chips, transistors, or otherdiscrete components. A module may also be implemented in programmablehardware devices such as field programmable gate arrays, programmablearray logic, programmable logic devices or the like.

Modules may also be implemented in software for execution by varioustypes of processors. An identified module of program instructions may,for instance, comprise one or more physical or logical blocks ofcomputer instructions which may, for instance, be organized as anobject, procedure, or function. Nevertheless, the executables of anidentified module need not be physically located together, but maycomprise disparate instructions stored in different locations which,when joined logically together, comprise the module and achieve thestated purpose for the module.

Furthermore, the described features, structures, or characteristics ofthe embodiments may be combined in any suitable manner. In the followingdescription, numerous specific details are provided, such as examples ofprogramming, software modules, user selections, network transactions,database queries, database structures, hardware modules, hardwarecircuits, hardware chips, etc., to provide a thorough understanding ofembodiments. One skilled in the relevant art will recognize, however,that embodiments may be practiced without one or more of the specificdetails, or with other methods, components, materials, and so forth. Inother instances, well-known structures, materials, or operations are notshown or described in detail to avoid obscuring aspects of anembodiment.

The description of elements in each figure may refer to elements ofproceeding figures. Like numbers refer to like elements in all figures,including alternate embodiments of like elements.

FIG. 1 is a schematic block diagram illustrating one embodiment of asystem 100 for observing and analyzing customer opinion. The system 100includes one or more cameras 102, a customer analysis apparatus 104, aplurality of persons 106 (106, as used herein may include one or morepersons); a plurality of products 108 a, 108 b, 108 c (collectively,“108”), a plurality of advertisements 110 a, 110 b, 110 c (collectively,“110”), and a focus 112 of a person 106, which are described below.

The system 100 includes one or more cameras 102 connected to a customeranalysis apparatus 104. The customer analysis apparatus 104 uses the oneor more cameras 102 to observe a plurality of persons 106 in thevicinity of a plurality of products 108 and/or advertisements 110. Theplurality of persons 106 may be individuals in a store, mall, cardealership, amusement park, convention center, museum, public space, orthe like. The products 108 may include food, clothing, automobiles,furniture, appliances, and/or other goods. The products 108 can bearranged in any number of configurations, including but not limited to alinear, circular, polygonal, and/or random manner. For example, theproducts 108 may be food items sitting on a shelf in a supermarket aisleor clothing on a circular clothes rack in a department store. As anotherexample, the products 108 may be automobiles parked in a showroom. Atany given time, the plurality of persons 106 may view a product (e.g.,108 a) from a number of different directions and angles, including fromall directions or only certain directions. For example, a person 106 canview items on a shelf or clothing in a display window from the front,and to a certain degree, from off to the side. At the same time, anautomobile on a showroom floor or a product display in a supermarketaisle may be viewed by a plurality of persons 106 from all directions.

An advertisement 110 may be placed next to a product 108 that is beingmarketed in the advertisement 110. Alternatively, an advertisement 110may be placed next to a product 108 that is unrelated to orcomplementary to the product 108 or service marketed in theadvertisement 110. For example, a selection of tortilla chips in asupermarket aisle may have an advertisement 110 for salsa displayed nextto it. As another example, a rack of suits in a department store mayhave an advertisement 110 for a tailoring service next to it. In oneembodiment, an advertisement 110 may appear by itself, such as anadvertisement on a store window pointed toward a street.

The one or cameras 102 are position to view the plurality of persons 106and products 108 from a variety of angles. Different types of cameras102 may be used within the system 100. A camera 102 may be a closedcircuit camera, a wireless camera, and/or a webcam. A camera's 102orientation may be fixed and thus only manually adjustable.Alternatively, a camera's 102 rotation may be remotely controlled by anoperator or automated using motion sensing technology. An example of aremote controlled camera is a PTZ (pan-tilt-zoom) camera commonly usedin casinos and hotels. In one embodiment, the one or more cameras 102may be positioned in the midst of an area containing a plurality ofproducts 108. For example, a PTZ camera may be placed in the ceiling ofa store or showroom to observe one or more customers. In anotherembodiment, the one or more cameras 102 may be placed around theperimeter of an area holding a plurality of products 108, such as one ormore corners of a store. In an alternative embodiment, a camera 102 maybe position to face in substantially one direction, such as in a storewindow display pointing outward in the direction of passersby.

The one or more cameras 102 may be directly or indirectly connected tothe customer analysis apparatus 104. For example, a camera 102 can bedirectly connected to the customer analysis apparatus 104 through one ormore cables or through a wireless connection using specific radiofrequencies. Alternatively, a camera 102 may be connected to thecustomer analysis apparatus 104 through a network, such as a local areanetwork (“LAN”), a wide area network (“WAN”), the internet, or awireless network (e.g., Wi-Fi, cellular, etc.). One of skill in the artwill recognize alternative ways that a camera 102 can be connected to acustomer analysis apparatus 104. Through the use of one or more cameras102, the customer analysis apparatus 104 is able to determine a focus112 of each person 106 in the vicinity of the plurality of products 108or advertisements 110. By determining the focus 112 of each person 106with respect to a product 108 or advertisement 110, the customeranalysis apparatus 104 can evaluate the interest of a person 106 withrespect to the product 108 or advertisement 110 as described withrespect to the apparatuses 200 and 300 of FIGS. 2 and 3, respectively.

FIG. 2 is a schematic block diagram illustrating one embodiment of anapparatus 200 for observing and analyzing customer opinion. Theapparatus 200 includes one embodiment of a customer analysis apparatus104 with a focus tracking module 202, an observation module 204, aninterpretation module 206, a rating module 208, a product trackingmodule 210, a guidance module 212, and an alert module 214, which aredescribed below.

The focus tracking module 202 uses the one or more cameras 102 todetermine a product 108 having a focus 112 of a person 106. A product108 is a good or service exchanged in a transaction, including but notlimited to, sales, rentals, leases, exchanges, or giveaways. A product108 can include a product 108 shown in an advertisement 110 or theadvertisement 110 itself. Sometimes, a product 108 may include acollection of goods, such as a rack of dress shirts. At other times, aproduct 108 may be a single item, such as a particular dress shirt in arack of dress shirts. A product 108 has a plurality of characteristics.Characteristics may include physical characteristics, such as theobject's size (e.g., height, length, width, circumference), weight,color, texture, composition (e.g., wood, metal, fiber, glass,combination), shape, finish (e.g., matte, polished), and othermeasurable physical traits. Characteristics may also refer tonon-physical attributes, such as origin (e.g., Made in Italy),manufacturer, brand name, product code, location, price, sales (e.g.,total number of product 108 sold during a period of time, total numberof product 108 returned during a period of time), and marketability.

The focus tracking module 202 analyzes video captured by the one or morecameras 102 to identify a product 108 having a focus 112 of the person106. The focus tracking module 202 may use video taken by multiplecameras 102 from different angles to more precisely determine theproduct 108 having a focus 112 of a person 106. The focus trackingmodule 202 may determine a product 108 having a focus 112 of a person106 in different ways. If a person's face is visible to the one or morecameras 102, the focus tracking module 202 can determine a product 108having a focus 112 of the person 106 by tracking the tilt of theperson's face and his or her gaze, including but not limited to thedirection of the person's pupils. If a person's face is not visible tothe one or more cameras 102, then the focus tracking module 202 maydetermine a product 108 having a focus 112 of the person 106 byextrapolating from one or more views of the person's head, including itstilt and rotation. For example, the focus tracking module 202 may find aproduct 108 having a focus 112 of a person 106 by extrapolating in adirection substantially opposite the back of the person's head.

The observation module 204 uses the one or more cameras 102 to monitorone or more physical, physiological, or behavioral characteristics of aperson 106 with respect to a product 108. The observation module 204 canalso use the one or more cameras 102 to monitor one or more physical,physiological, or behavioral characteristics of multiple persons 106. Aperson's physical characteristics include, but are not limited to,height, weight, gender, body type, eye color, skin tone, hair color,skin condition (e.g., smooth, wrinkled), tattoos, or presence/absence ofhair (e.g., facial hair and/or baldness). A person's physiologicalcharacteristics include, but are not limited to, breathing pattern, thepresence/absence of perspiration, body temperature, the presence/absenceof skin color (e.g., if the person's face is flush), heart rate, andpupil dilation. A person's behavioral characteristics include, but arenot limited to, body language, speech, facial expressions, eyemovements, and gestures. Examples of a behavioral characteristic iswhether a person is touching a product 108 or holding a product 108 upclose.

In one embodiment, the observation module 204 monitors one or morephysical, physiological, or behavioral characteristics of a person 106with respect to a product 108 by visually recording the person 106 withthe one or more cameras 102 over a period of time. The observationmodule 204 may monitor the person 106 for a preset amount of time (e.g.,first five seconds after determining the person's product 108), a subsetof time that a person 106 is focused on a product 108, the entire time aperson 106 is focused on a product 108, or after the person's focus hasmoved away from the product 108. An example of the observation module204 monitoring one or more physical, physiological, or behavioralcharacteristics of a person 106 is using one or more cameras 102 torecord a person's facial expressions or subtle changes in the color ofthe person's face. Another example is the observation module 204 usingone or more cameras 102 to record a movement of a person's pulse in hiswrist, neck, or other body part.

In another embodiment, the observation module 204 monitors the one ormore physical, physiological, or behavioral characteristics of a person106 by using other apparatuses in conjunction with the camera 102. Forexample, the observation module 204 may use an infrared detector todetermine a person's skin temperature and/or whether the person's skinis flush. The infrared detector may be thermal or photonic. Other typesof apparatuses that may be used in conjunction with the one or morecameras 102 to monitor one or more physical, physiological, orbehavioral characteristics of a person 106 include scanners that detectelectromagnetic radiation (e.g., millimeter wave scanner with active orpassive sensors) or microphones that pick up sounds from a person 106(e.g., words, accents, speech patterns, laughter, sobbing).

The interpretation module 206 determines an opinion of a person 106 withrespect to the product 108 being focused on by the person 106. Anopinion may be positive, negative, or neutral. An opinion has anemotional component. Examples of emotions include, but are not limitedto, fear, anger, sadness, joy, disgust, trust, anxiety, anticipation,surprise, and excitement. Sometimes, an opinion can include acombination of emotions. For example, a negative opinion may includeboth anger and disgust. An opinion also has an intensity, which can bemeasured and expressed in a variety of forms. For example, an opinionmay be expressed on a numerical scale of 1 to 10, as different colors,and/or different sized shapes. The overall intensity of an opinion maybe a function of the intensity of each emotion associated with theopinion. For example, a moderately positive opinion can be the result ofgreat joy tempered with moderate sadness.

In an embodiment, the interpretation module 206 determines an opinion ofa person 106 with respect to the product 108 by comparing the one ormore physical, physiological, or behavioral characteristics recorded bythe one or more cameras 102 to physical features, physiologicalsymptoms, or behaviors mapped to specific emotions or opinions. Forexample, the interpretation module 206 may determine that a customer hasa positive opinion of an advertisement 110 when it analyzes thecustomer's facial expression while the customer is looking at theadvertisement 110 and notices that the customer is smiling.Alternatively, the interpretation module 206 may determine that acustomer has a negative opinion of a product 108 when it analyzes thecustomer's facial expression while the customer is holding the product108 and notices that the customer has a look of disgust and/or sadness.As another example, the interpretation module 206 may determine that acustomer has a highly positive opinion of a product 108 when it analyzesthe customer's speech patterns and notes that the customer is speakingexcitedly. In yet another example, the interpretation module 206 maydetermine a customer's opinion of a product 108 by analyzing thecustomer's posture (e.g., leaning in or away from the product 108), armposition (e.g., crossed in front of chest, holding the product 108 upclose), the length of time that the customer has been focused on theproduct 108 (e.g., longer focus denotes more interest and higherpositive opinion), or physiology (e.g., quickened pulse, flush face,rapid breathing, pupil dilation may indicate excitement, surprise, orjoy).

In one embodiment, the interpretation module 206 determines an opinionof a person 106 with respect to a product 108 over a period of time.This period of time may be predetermined (e.g., two second interval) ordenoted by the amount of time that a person remains focused on theproduct 108. It may be advantageous to determine an opinion of a person106 with respect to a product 108 over time because the person's opinionmay change during that time. For example, a customer who sees a pair ofshoes may initially have a positive reaction or opinion of the shoes andquickly walk over to pick up the shoes. But the customer's opinion ofthe shoes may change to become negative once the customer sees the priceof the shoes. The opinion of a person 106 with respect to a product 108over a period of time may be the person's initial opinion at the startof the period of time or the person's final opinion at the end of theperiod of time. Alternatively, the opinion of a person 106 with respectto the person's product 108 over a period of time may be a combination(e.g., average) of the person's opinions during that period of time.

In some embodiments, the interpretation module 206 also determines thata person 106 needs assistance to find a product 108. In this instance,the interpretation module 206 determines that a person 106 needsassistance to find a product 108 by matching the one or more behavioralcharacteristics recorded by the one or more cameras 102 to behaviorsindicating that a person 106 needs assistance. For example, theinterpretation module 206 may determine that a customer needs assistancewhen it recognizes that he is scanning a rack of clothes or the store bymoving his eyes and/or head. In certain embodiments, the interpretationmodule 206 also determines that a person 106 needs assistance withrespect to a product 108. In this instance, the interpretation module206 determines that a person 106 needs assistance with respect to aproduct 108 by matching the one or more behavioral characteristicsrecorded by the one or more cameras 102 to behaviors indicating that aperson 106 needs assistance with respect to a product 108. For example,the interpretation module 206 may determine that a customer needs helpto find a particular size of clothing when it recognizes that thecustomer is digging through a pile of clothes.

In some instances, the interpretation module 206 also determines that aperson 106 poses a potential threat. In this case, the interpretationmodule 206 determines that a person 106 poses a potential threat bymatching the one or more behavioral characteristics recorded by the oneor more cameras 102 to behaviors indicating that a person 106 may harmanother person 106 or damage property. For example, the interpretationmodule 206 may determine that a customer is a potential threat when itrecognizes that the customer has hidden a product 108 in her purse anddetermines that the customer's heart rate rose dramatically. In such asituation, the one or more cameras 102 may automatically zoom in on thecustomer and continue to follow him or her throughout the store.

In a certain embodiment, the interpretation module 206 may determine amedical condition of a person 106. A medical condition of a person 106can include a variety of conditions, such as a heart attack, stroke, orrespiratory problems (e.g., asthma attack). The interpretation module206 may determine a medical condition of a person 106 by analyzing theone or more physical, physiological, or behavioral characteristicsrecorded by the one or more cameras 102. For example, the interpretationmodule 206 may determine that a person 106 has a respiratory problem bycomparing the person's breathing pattern and/or sounds to known patternsor sounds indicating respiratory problems. As another example, theinterpretation module 206 may determine that a person 106 is having aheart attack by detecting the presence of a cold sweat and shortness ofbreath in the video recorded by the one or more cameras 102 and matchingit to known patterns.

The rating module 208 adjusts one or more ratings of the product 108 inresponse to the interpretation module 206 determining the opinion of theperson 106 regarding the product 108. Each characteristic for a product108 may have a rating, and the product 108 may have a composite ratingthat is a function of individual ratings for each characteristic of theproduct 108. The composite rating may be a simple average of all theindividual ratings, a weighted average, or a similar average of a subsetof the individual ratings. A rating may be expressed or measured in avariety of forms. For examples, it can be expressed on a numerical scaleof 1 to 100 or one to five stars. Initially, a product 108 may have anaverage rating (e.g., 50 out of 100) that is then adjusted upward ordownward based on the opinion of a person 106. A positive opinion from aperson 106 will cause the rating of a product 108 to rise, while anegative opinion from a person 106 will cause the rating to fall. Theintensity of an opinion from a person 106 determines how much the ratingof a product 108 may rise or fall.

In one embodiment, the rating module 208 adjusts the composite rating ofthe product 108 in response to the interpretation module 206 determiningthe opinion of the person 106 regarding the product 108. In anotherembodiment, the rating module 208 adjusts a rating of one or morecharacteristics of the product 108 in response to the interpretationmodule 206 determining the opinion of the person 106 regarding theproduct 108. For example, the interpretation module 206 may determinethat a customer's initial opinion to seeing a product 108 in a retailstore is positive, which may cause the rating module 208 to adjust theratings for the physical characteristics of the product 108 upwards. Butif the interpretation module 206 determines a few seconds later that thecustomer's opinion changes over time, the rating module 208 may thenadjust the ratings for the physical characteristics of the product 108downwards.

The product tracking module 210 stores the one or more ratings of theproduct 108. The product tracking module 210 may store the one or moreratings of the product 108 using a relational database managementsystem. In an embodiment, the product tracking module 210 stores onlythe most recent composite rating and rating for each characteristic ofthe product 108. In another embodiment, the product tracking module 210stores a history of ratings for a product 108. For example, the producttracking module 210 can store the daily composite rating of a product108 for the past week, or the hourly composite rating of the price of aproduct 108 for the past day. Where the product tracking module 210stores a history of ratings for a product 108, it can also store arunning average of each rating (e.g., average composite rating for aproduct 108 during the past week, average price rating for a product 108during the past month). The product tracking module 210, in oneembodiment, also stores a number of persons 106 who have placed theirfocus 112 on a specific product 108. In a further embodiment, theproduct tracking module 210 also stores a total amount of time that aproduct 108 has been a focus 112 of one or more persons 106 during aperiod of time.

The guidance module 212 recommends one or more actions regarding theproduct 108 based on the one or more ratings of the product 108. Forexample, the guidance module 212 can recommend that a store stock moreof a product 108 when the composite rating of the product 108 meets orexceeds a specific threshold. Conversely, the guidance module 212 canrecommend that a store stop selling a product 108 or switchadvertisements 110 for the product 108 in response to a steep drop inthe product 108 or the advertisement's composite rating or price rating.Alternatively, the guidance module 212 may recommend that a store lowerthe price of a product 108 when the moving average of the product'scomposite rating or price rating is trending lower. In another example,the guidance module 212 may recommend that a store change the locationof a product 108 within the store based upon its composite rating. Theguidance module 212 may also recommend extending a sale or promotion inan advertisement 110 based upon the composite rating of theadvertisement 110. In some cases, the guidance module 212 may recommendthat a product 108 be moved to another location because it has a lowlocation rating. Alternatively, the guidance module 212 may recommendchanging the configuration of the store based on the location ratings ofcertain products 108.

In some cases, the guidance module 212 recommends one or more actionsregarding the product 108 based on a number of persons who have focusedon the product 108 during a period of time. In other cases, the guidancemodule 212 recommends one or more actions regarding the product 108based on an amount of time that the number of persons have focused onthe product. For example, the guidance module 212 may recommend that apair of shoes in a store be moved to a different location in the storeif very few people have focused on the shoes in a week. As anotherexample, the guidance module 212 may recommend that a store order moreof a certain product 108 because customers spend a lot of time lookingat them.

The alert module 214 sends a notification to one or more other persons106 in response to the interpretation module 206 determining a medicalcondition of the person 106. The one or more other persons 106 may bemedical personnel (e.g., doctors, nurses, and/or EMTs), store employees,and/or security guards. The notification may take the form of a textmessage, electronic mail, an audible warning (e.g., siren), and/or avisual warning (e.g., flashing light, strobe light). The notificationmay include the person's description or location. For example, the alertmodule 214 may send a text message to store employees notifying them acustomer on the 3^(rd) floor of the department store may need medicalattention while sending a similar notification to an emergency callcenter (e.g., 911).

In one embodiment, the alert module 214 also sends a notification to oneor more other persons 106 in response to the interpretation module 206determining that the person 106 needs assistance to find a product 108.For example, the alert module 214 may send a text message to asupermarket employee that a customer may need assistance finding aproduct 108 in a particular supermarket aisle. In another embodiment,the alert module 214 also sends a notification to one or more otherpersons 106 in response to the interpretation module 206 determiningthat the person 106 needs assistance with respect to a product 108. Forexample, the alert module 214 may notify store employees to check thestore inventory when it detects a customer looking through a rack ofclothes for a particular style of clothing.

In some embodiments, the alert module 206 sends a notification to one ormore other persons 106 in response to the interpretation module 206determining that a person 106 is a potential threat. The alert module206 may notify store employees and/or a security service with theperson's description and/or location. For example, the alert module 214may send a text message to security guards notifying them that acustomer in a certain aisle is acting suspiciously.

FIG. 3 is a schematic flow chart diagram illustrating one embodiment ofa method 300 for observing and analyzing customer opinion. The method300 begins and using one or more cameras 102 determines 302 a product108 having a focus 112 of a person 106, the product 108 having aplurality of characteristics. The method 300 uses the one or morecameras 102 to monitor 304 one or more physical, physiological, orbehavioral characteristics of the person 106 with respect to the product108. The method 300 determines 306 an opinion of the person 106regarding the product 108. The method 300 adjusts 308 one or moreratings of the product 108 in response to determining the opinion of theperson 106 regarding the product 108, and the method 300 ends.

FIG. 4 is a schematic flow chart diagram illustrating another embodimentof a method for observing and analyzing customer opinion in accordancewith the present invention. The method 400 begins and using one or morecameras 102 determines 402 a product 108 having a focus 112 of a person106, the product 108 having a plurality of characteristics. The method400 uses the one or more cameras 102 to monitor 404 one or morephysical, physiological, or behavioral characteristics of the person 106with respect to the product 108. The method 400 determines 406 anopinion of the person 106 regarding the product 108. The method 400adjusts 408 one or more ratings of the product 108 in response todetermining the opinion of the person 106 regarding the product 108. Themethod 400 stores 410 the one or more ratings of the product 108, thenumber of persons 106 who have focused on the product 108, and theamount of time that the number of persons 106 have focused on theproduct 108. The method 400 recommends 412 one or more actions regardingthe product 108 based on the one or more ratings of the product 108, andthe method 400 ends. In one embodiment, the method 400 also recommends412 one or more actions based on a number of persons 106 who havefocused on the product 108 and/or an amount of time that the number ofpersons 106 have focused on the product 108, and the method 400 ends. Inanother embodiment, the method 400 modifies 414 the recommended actionbased on a number of persons 106 who have focused on the product and anamount of time the number of persons 106 have focused on the product,and the method 400 ends. For example, one may recommend differentactions for two products 108 with similar ratings if the first product108 has been viewed by a lot more customers than the second product 108.In this example, the rating for the first product 108 is more likely tobe accurate than the rating for the second product 108 because of thelarger sample size of customers who have viewed the first product 108.

FIG. 5 is a schematic flow chart diagram illustrating yet anotherembodiment of a method for observing and analyzing customer opinion inaccordance with the present invention. The method 500 begins and usingone or more cameras 102 determines 502 a product 108 having a focus 112of a person 106, the product 108 having a plurality of characteristics.The method 500 uses the one or more cameras 102 to monitor 504 one ormore physical, physiological, or behavioral characteristics of theperson 106 with respect to the product 108. The method 500 determines506 an opinion of the person 106 regarding the product 108. The method500 adjusts 508 one or more ratings of the product 108 in response todetermining the opinion of the person 106 regarding the product 108. Inone embodiment, the method 500 determines 510 a medical condition of theperson 106, and in response, sends 516 a notification to one or moreother persons 106, and the method 500 ends. In another embodiment, themethod 500 determines 512 that the person 106 needs assistance regardingthe product 108, and in response, sends 516 a notification to one ormore other persons, and the method 500 ends. In certain embodiments, themethod 500 determines 514 that the person 106 is a potential threat, andin response, sends 516 a notification to one or more other persons, andthe method 500 ends.

The embodiments may be practiced in other specific forms. The describedembodiments are to be considered in all respects only as illustrativeand not restrictive. The scope of the invention is, therefore, indicatedby the appended claims rather than by the foregoing description. Allchanges which come within the meaning and range of equivalency of theclaims are to be embraced within their scope.

What is claimed is:
 1. A system comprising: one or more cameras; and an analysis apparatus coupled to the one or more cameras, the analysis apparatus comprising: a focus tracking module that uses the one or more cameras to identify an actual consumer product displayed in a real-world retail environment having a focus of a customer, said focus on the consumer product determined by: tracking face tilt and eye gaze of the customer in relation to the consumer product when the customer's face is visible to the one or more cameras, and tracking head tilt and rotation of the customer in relation to the consumer product when the customer's face is not visible to the one or more cameras; an observation module that uses the one or more cameras to detect, from a distance, one or more physiological characteristics of the customer with respect to the consumer product; an interpretation module that determines an opinion of the customer regarding the consumer product based on the detected one or more physiological characteristics; and a rating module that adjusts a set of ratings of the consumer product in response to the interpretation module determining the opinion of the customer regarding the consumer product, wherein at least a portion of the observation module, the focus tracking module, the interpretation module, and the rating module comprise one or more of computer hardware and executable code, the executable code stored on one or more non-transitory computer-readable storage media.
 2. The system of claim 1, further comprising a product tracking module that stores the one or more ratings of the consumer product, a number of customers who have focused on the consumer product, and an amount of time that the number of customers have focused on the consumer product.
 3. The system of claim 2, further comprising a guidance module that recommends one or more actions regarding the consumer product based on the one or more ratings of the consumer product.
 4. The system of claim 3, wherein the guidance module further recommends one or more actions regarding the consumer product based on a number of persons who have focused on the consumer product and an amount of time that the number of persons have focused on the consumer product.
 5. The system of claim 1, wherein the interpretation module further determines a medical condition of the customer, and further comprising an alert module that sends a notification in response to the interpretation module determining medical condition of the customer.
 6. The system of claim 1, wherein the interpretation module further determines that the customer needs assistance with respect to the consumer product, and further comprising an alert module that sends a notification in response to the interpretation module determining that the customer needs assistance.
 7. A method comprising: using one or more cameras to identify an actual consumer product being displayed in a real-world retail environment having a focus of a person, said focus on the consumer product determined by: tracking face tilt and eye gaze of the customer in relation to the consumer product when the customer's face is visible to the one or more cameras, and tracking head tilt and rotation of the customer in relation to the consumer product when the customer's face is not visible to the one or more cameras; using the one or more cameras to detect, from a distance, one or more physiological characteristics of the customer with respect to the consumer product; determining, by a processor, an opinion of the customer regarding the consumer product based on the detected one or more physiological characteristics; and adjusting one or more ratings of the consumer product in response to the processor determining the opinion of the customer regarding the consumer product.
 8. The method of claim 7, further comprising storing the one or more ratings of the consumer product, a number of customers who have focused on the consumer product, and an amount of time that the number of customers have focused on the consumer product.
 9. The method of claim 8, further comprising recommending one or more actions regarding the consumer product based on the one or more ratings of the consumer product.
 10. The method of claim 9, further comprising recommending one or more actions regarding the consumer product based on a number of customers who have focused on the consumer product and an amount of time that the number of customers have focused on the consumer product.
 11. The method of claim 7, further comprising determining a medical condition of the customer, and further comprising an alert module that sends a notification in response to the interpretation module determining a medical condition of the customer.
 12. The method of claim 7, further comprising determining that the customer needs assistance with respect to the consumer product, and further comprising an alert module that sends a notification in response to the interpretation module determining that the customer needs assistance.
 13. A computer program product for observing and analyzing customer opinion, the computer program product comprising a non-transitory computer-readable storage medium having program instructions embodied therewith, the program instructions readable/executable by a processor to cause the processor to: use one or more cameras to identify an actual product being displayed in a real-world retail environment having a focus of a customer, said focus on the consumer product determined by: tracking face tilt and eye gaze of the customer in relation to the consumer product when the customer's face is visible to the one or more cameras, and tracking head tilt and rotation of the customer in relation to the consumer product when the customer's face is not visible to the one or more cameras; use the one or more cameras to detect, from a distance, one or more physiological characteristics of the customer with respect to the consumer product; determine an opinion of the customer regarding the consumer product based on the detected one or more physiological characteristics; and adjust one or more ratings of the consumer product in response to the interpretation module determining the opinion of the customer regarding the consumer product.
 14. The computer program product of claim 13, the program instructions further causing the processor to store the one or more ratings of the consumer product, a number of persons who have focused on the product, and an amount of time that the number of customers have focused on the product.
 15. The computer program product of claim 14, the program instructions further causing the processor to recommend one or more actions regarding the consumer product based on one of the one or more ratings of the consumer product, a number of customers who have focused on the consumer product, and an amount of time that the number of customers have focused on the consumer product.
 16. The computer program product of claim 13, the program instructions further causing the processor to determine a medical condition of the customer, and further comprising an alert module that sends a notification in response to the interpretation module determining medical condition of the customer.
 17. The computer program product of claim 13, the program instructions further causing the processor to determine that the customer needs assistance with respect to the consumer product, and further comprising an alert module that sends a notification in response to the interpretation module determining that the customer needs assistance.
 18. The system of claim 1, wherein the one or more physiological characteristics detected from the distance comprises one of a pulse, heart rate, breathing rate, breathing pattern, skin temperature, body temperature, and pupil dilation.
 19. The system of claim 1, wherein: the consumer product comprises a plurality of characteristics; and the set of ratings comprises a respective rating for each characteristic of the plurality of characteristics.
 20. The system of claim 1, wherein the set of rating further comprises a composite rating based on the respective rating for each characteristic of the plurality of characteristics. 